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ABSTRACT 

The results of a course-instructor survey (CIS) to 
assess student perceptions of instructional effectiveness are made 
available to instructors on the assumption that feedback results in 
appropriate changes in instructional behavior. The purpose of this 
study was to investigate the effects of feedback of CIS results to 
instructors. Two specific questions provided the basis of the study: 

(a) Does CIS feedback positively affect subsequent student r?\tings? 

(b) Does the content of the scale influence the likelihood of change 
for instructors receiving feedback about the scale? Twenty-nine 
instructors in junior and senior level education courses at the 
University of Texas at Austin participated in the study, with the 
number of students in their classes ranging from 25-45. To obtain 
scores for feedback to instructors, the CIS was administered at 
midsemester and during the last week of the semester. At midsemester, 
the instructors also completed the CIS for their own teaching as they 
believed students perceived it. Fifteen instructors received CIS 
information at midsemester, and 14 instructors who received no 
feedback of CIS results constituted the control group. Results of the 
study did not support the general effectiveness of CIS feedback. A 
table containing the highest loading items on the CIS factor and a 
bibliography are attached. (JS) 
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Use of a cours: -instructor survey (CIS) to assess student percep- 
tions of instructional effectiveness is widespread. Typically, the 
results of such surveys are made available to the instructors, apparently 
with the implicit assumption that such feedback will result in appropriate 
changes in instructional behavior. Several studies have been conducted 
to determine whether such feedback indeed does result in improved instruc- 
tion (as measured by the students' ratings), but the findings of these 
studies have been mixed. On the positive side. Gage, Runkel, and Chat- 
terjee (1963) found that when pupil ratings of their "ideal" teacher 
were fed back to teachars, subsequent teacher behaviors were perceived 
by the pupils as moving closer to the ideal ratings, compared to a control 
group which did not receive the ratings. In a study by Tuckman and Oliver 
(1968), perceived teacher behavior was more positive when pupil ratings of 
the teachers were used as feedback, compared to supervisor ratings or to no 
feedback of ratings. Studies showing no effects for feedback include one 
by Miller (1971) and one by Centra (1972). Both studies compared groups 
of teachers receiving feedback of student ratings to no feedback groups. 
Centra (1972) did find, however, that some teachers who received feed- 
back did change positively. Generally, these were teachers who had ini- 
tially expected to receive more positive ratings than they actually received; 
from their students. 

The purpose of the present study was to further investigate the 
effects of feedback of CIS results to instructors. Specifically, the fol-* 
lowing questions were examined! Does CIS feedback positively affect sub- 
sequent student ratings? Does the instructor's accuracy in perceiving 
how his students rate him influence the Ixkllhood of obtaining subsequent 
Improved ratings? Does the content of the scale influence the liklihood 
of change for instructors receiving feedback about the scale? In other 
words, feedback on some types of items might have an effect, whereas feed- 
back concerning other items might be of no consequence. It was expected 
in this study that if significant effects for feedback were found, they 
would be on scales whose items represented concrete, behavioral character- 
istics rather than scales whose items reflected more the personal attri- 
\: butes of the instructor* 

•^5^1 Presented at the Annual meeting of the American Educational Research 

Association, April 15-19^ 1974, Chicago, IL. 
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Procedures 

The participants in this study were 29 instructors in junior and 
senior level education courses at the University of TexavS at Austin, 
Eight of the instructors held faculty rank; the remaining: twenty-one 
were teaching assistants. The courses represented in the sizudy were 
educational psychology, general methods, and several special methods 
courses. All participants were assured confidentiality prior to their 
participation. The number of students in the various, classes was mod- 
erate, with no class exceeding 45, and most classes having numbers in 
the 25-35 range. 

The questionnaire used in this study was the Course Instructor Sur- 
vey (CIS): Student Questionnaire , form 2, made available through the 
Measurement and Evaluation Center of The University of Texas at Austin. 
The CIS consists of 40 items, 35 of which require students to respond on 
a four point scale about their perceptions of the course, the instructor > 
and his teaching activities. The form used was based in part upon sug- 
gestions from a committee of students in the College of Education,, and 
some items reflect the concerns of this population (e.g. "The instructor 
seemed genuinely interested in making me an effective teacher"). However, 
most items are applicable to instruction in all areas of undergraduate 
college courses. 

To obtain CIS scores for feedback to instructors, the CIS was adminis- 
tered in each of the '19 classes at midsemester and during the last week of 
the semester by student proctors. The instructors were not present during 
the administrations of the CIS. Students were informed that the purpose 
of the midsemester administration was to provide some of the instructors 
feedback about their course before the semester ended. Students were also 
told that since their instructor might see their responses before assign- 
ing grades they should disguise their handwriting, if they wished to add 
any comments. 

One of the purposes of the study was to determine whether an instruc- 
tor's perception of his teaching behavior, relative to his students' per- 
ceptions, might influence feedback effectiveness. To provide a basis for 
testing this hypothesis, instructors were asked, at midsemester, to com- 
plete the CIS for their own teaching, as they believed students perceivv^d 
it. The CIS was completed prior to any feedback. 

Within two weeks of the midsemester fiurvey in their classes, 15 instruc- 
tors, randomly chosen from the total group, were given the results of the 
CIS in their classes. The group of 15 itts1;ructors receiving the CIS infor- 
mation at midsemester were designated as the Feedback group. The results 
transmitted as feedback Included the meati and S*D. of the class response to 
each item, the instructor's percentile ranking on the item based on norms 
from a large number of classes surveyed the preceeding semester, and the 
frequency of response to each item alternative. These results were those 
generally made available to instructors participating in the CIS end*-of- 
semester survey. The remaining 14 Instructors who received no feedback of 
the CIS reaults constituted the control group for the study* 
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Results 

The intercorrelations among student rei^iponses to the 35 items from 
the CIS were factor analyzed, using the principal components solution with 
rotation to the varimax criterion. Six factors were identified. The 
factors, and the items most highly correlated with each factor are listed 
in Table 3, Scores on each factor for each student were computed, and 
from these factor ^ 'ores, means for each class were obtained. The mean 
change from the midsemester to the final administration for each of the 
six factors was the unit of analysis for the tests of hypotheses. 

In order tp test the hypothesis that the amount of discrepancy between 
perceived and actual ratings affects subsequent behavior, the instructors 
were divided into three discrepancy groups for analysis. The CIS completed 
by each instructor at midsemester was scored on each of the six factors 
listed in Table 1. The discrepancy between the instructor *s score and the 
midsemester class mean was then computed. For each of the six variables, 
instructors were divided into three discrepancy groups according to whether 
they believed that (1) students would perceive them more positively than 
they actually did (Positive discrepancy); (2) students would perceive them 
similarly to what they actually did (Low discrepancy); or (3) students 
would perceive them more negatively than they actually did (Negative dis- 
crepancy) . 

The data were analyzsed using a 2 (feedback - no feedback) x 3 (levels 
of discrepancy) ANOVA design. Each of the six CIS variables was analyzed 
separately. 

The hypothesis that feedback of CIS results has a general effect on 
subsequent instructor behavior was tested by the main effect for the feed- 
back-no feedback dimension. In none of the analyses was the F ratio signi- 
ficant at the .05 level. Thus, there was no support for this hypothesis. 

The hypothesis that the effect of feedback is a function of the degree 
or type of discrepancy between the classes' ratings and the instructor's per- 
ception was tested by the interaction effect of the feedback-no feedback 
dimension and the discrepancy dimension ♦ On five of the six factors there 
were no significant interactions. On one of the factors. Adequacy pj^^ Ev alu a- 
tion, the interaction was significant (F « 4.16; p < .05, df « 2,23). Com- 
pared to the no feedback groups, instructors in the Positive discrepancy 
and the Low discrepancy groups received higher ratings after feedback, 
whereas Negative discrepancy instructors changed least. This result is 
consonant with the hypothesis since the Negative discrepancy instructors 
are those who received midsemester ratings on this factor higher than they 
expected, and who thus would have least motivation to alter their behavior. 
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The hypotheses that feedback of CIS results has a geiis^ral effect on 
subsequent instructor ratings was not supported • There was some support 
tor the hypothesis that change in CIS ratings is a function of the item 
content and of the discrepancy between the instructor's perception^and 
the class rating. On the scale ''Adequacy of Evaluation" more change was 
observed among instructors who were not rated as well as they had expected, 
than among instructors who received feedback that was more positive than 
they anticipated. It is also worth noting that "Adequacy of Evaluation" 
contains items which reflect quite specific behaviors. Thus, changes on 
this variable might be a result of an instructor's ability to identify 
readily those behaviors that reflect a low rating and to alter his behavior 
accordingly. Such a change might also be more easily identified by stu- 
dents, since the behavioral referents for the items are presumably as 
obvious to them as to the instructor. 

On most of the other factors, the items are ler>s descriptive of 
instructor beha^^ior, and sometimes simply indicate student behaviors or 
feelings rather than those of the instructor. The apparent exception is 
the factor "Instructor Disinterest," which identifies instructor behaviors 
at approximately the same level of specificity as "Adequacy of Evaluation." 
Since no significant effects were obtained for "Instructor Disinterest," 
the hypothesis that change in instructor behavior is related to both dis- 
crepancy and item content can be advanced only very tentatively, although 
the arguments in its favor are plausible. From a common-sense point of 
view, the greater the specificity with which instructiotial behaviors are 
identified during feedback, the easier it would be for an instructor to 
identify what should be changed. From a more theoretical perspective, 
Schmuck (1971) suggests that teacher self-confrontation procedures result 
in anxiety which could lead to one or a combination of the following coun- 
terproductive defensive strategies: (1) perceiving "ideal" performance 
states as unrealistic, (2) perceiving information about actual performances 
as invalid J (3) perceiving information about discrepancies between ideal 
and actual performances as being typical for all teachers, or (4) per- 
ceiving actual performances as pursuing unstated goals. Def ensiveness 
could conceivably obviate any classroom performance feedback provided to 
an instructor. However, such defensive behavior, while influencing recep- 
tivity to feedback for all types of items, might be less pronounced for 
a specific, behaviorally referenced item than for items with less obvious 
behavioral referents. 

That the present study does not support the general effectiveness of 
CIS feedback is consistent with findings from Centra (1972) and Miller 
(1971), which were conducted in college settings. On the other hand the 
results conflict with the findings of Gage, et al (1963) and Tucktnan and 
Oliver (1968), which were conducted with elementary and secondary teachers > 
respectively. It is possible that receptivity to feedback is included 
among the many variables differentiating these groups of teachers* 

A tentative recommendation that can be made from results of this study 
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is to include in counie-instructor evaluation questionnaires, items which 
are as behaviorally referenced as possible, presuming that change in 
instructor behavior is a desirable outcome of the feedback process. Other 
feedback modes could also be experimented with, perhaps in conjunction with 
the traditional CIS feedback procedures. 
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Table 1. Highest loading items on each of six CIS factors. 



Factor Item Loading 

Instructional He presented the material coherently, .80 

Effectiveness emphasizing the major points and making 

clear their relationships. 

The instructor seemed to be well-pre .76 
pared for lecture or discussion. 

The examples and illustrations used .75 
made Lhe material clearer to me. 

He usually held my attention during .74 
class. 

Given the opportunity I would probably .72 
choose this instructor again for 
another course. 

He was intellectually stimulating. .70 

I enjoyed attending class. .67 

He usually was aware of whether the .67 
class members were following his 
discussion or lecture with under- 
standing. 

I found this course to be interesting. .62 



Instructor The instructor often failed to come .67 

Disinterest to class without previous notifi- 

cation. 

The instructor referred to his .65 
experiences too often. 

The instructor had annoying habits .61 
which were distracting. 

Adequacy of He had sufficient evidence in terms .71 

Evaluation of class participation and written 

work, to evaluate my achievement. 

He commented individually on my .65 

written work, either orally or in 

writing. 

He usually returned tests and assign- .58 
ments promptly. 



ERIC 



Table 1, cont'd. 



The meaning of questions on his 
tests were usually clear. 



.55 



Student 

Involvement 



I made an honest effort to learn 
in this course. 

I learned much material applicable 
to my future work. 



.68 



.55 



Receptivity 
To Students 



He made me feel free to ask questions, ,68 
disagree ) and express my ideas. 

He was fair and impartial in his ♦Gtt 
dealings with students. 

The instructor in his dealings .66 
with students seemed to respect 
them as individuals. 



Amount of 

Work/Difficulty 



In my opinion the class assignments 
were too time-consuming. 

I thought this course was unusually 
difficult for me. 



-.69 
•.67 
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